#model capacity

Meta and collaborators developed a novel framework to accurately quantify how much language models memorize training data, estimating GPT models store around 3.6 bits per parameter and providing new insights into memorization versus generalization.